Visualizing sentiment results with visual indicators representing user sentiment and level of uncertainty

ABSTRACT

Sentiment analysis of user feedback is performed, using uncertainty rules. Different levels of uncertainty associated with sentiment results of the sentiment analysis are determined, where the sentiment results identify user sentiments contained in the user feedback. At least one visualization of the sentiment result is provided, where the visualization has first visual indicators to represent respective user sentiments, and second visual indicators associated with the first visual indicators to represent respective levels of uncertainty associated with the respective user sentiments.

BACKGROUND

Users often provide feedback, in the form of reviews, regardingofferings (products or services) of different enterprises. As examples,users can be external customers of an enterprise, or users can beinternal users within the enterprise. An enterprise may wish to usefeedback to improve their offerings. However, there can be potentially avery large number of received reviews, which can make meaningfulanalysis of such reviews difficult and time-consuming.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

Some embodiments are described with respect to the following figures:

FIG. 1 is a block diagram of an example arrangement incorporating someimplementations;

FIGS. 2-5 illustrate different example visualizations according tovarious implementations;

FIG. 6 is a flow diagram of an example containing visual analysistechniques according to some implementations; and

FIG. 7 is a block diagram of an example system incorporating a sentimentcertainty visual analysis mechanism according to some implementations.

DETAILED DESCRIPTION

An enterprise can receive relatively large amounts of data, such as userfeedback, in the form of comments. The comments can be received over anetwork, such as the Internet or a local network, where users (externalusers and/or internal users of the enterprise) can supply commentsregarding a product or service through the enterprise's website or otheraccessible location, or through a third party website such as a socialnetworking site. Alternatively, or additionally, comments can bereceived in paper form and entered by the enterprise's personnel into asystem in electronic form. A “comment” or “user comment” can refer to anentire record containing a given user feedback, or some portion of therecord (e.g. a sentence, paragraph, or other section of the record).

In some cases, it may be desirable to visually analyze received commentsby employing automated visualization of the comments in graphical form(without a user having to read the individual comments). When there is arelatively large number of comments, however, graphical elementsrepresenting corresponding comments can be close to each other or canactually overlap each other, particularly when the comments areassociated with the same time points or time points that are relativelyclose to each other. A large number of overlapping graphical elements orgraphical elements close to each other can make it difficult tounderstand what is being represented by the graphical elements.

Analysis of user comments can allow for better understanding of usersentiment regarding an offering of an enterprise. An offering caninclude a product or a service provided by the enterprise. A “sentiment”refers to an attitude, opinion, or judgment of a human with respect tothe offering.

With a relatively large number of comments and associated terms that canappear in such comments, there can be uncertainty in sentiment analysisresults. Uncertainty in sentiment analysis results can produceinaccurate information that can cause an enterprise to take incorrectactions to address issues that may be identified by the sentimentanalysis.

In accordance with some implementations, uncertainty rules (which canalso be referred to as “certainty rules”) are provided to allow adetermination of levels of uncertainty associated with sentiment resultsproduced from a sentiment analysis. FIG. 1 illustrates an example systemfor performing sentiment certainty visual analysis. User comments 102(which can be included in documents or records) are received by asentiment certainty visual analysis module 104. The sentiment certaintyvisual analysis module 104 analyzes each comment 102 to identify partsof the text within the comment that includes attributes that can beassociated with sentiment words. An “attribute” can refer to a noun orcompound noun (a noun formed of multiple words, such as “customerservice”) that exists in a comment. In other examples, an attribute canalso include a verb or adjective, or some combination of a noun, verb,or adjective.

Sentiment words in the comment can also be identified, where sentimentwords include individual words or phrases (made up of multiple words)that express an attitude, opinion, judgment of a human. Examples ofsentiment words include “bad,” “poor,” “great performance,” “fastservice,” and so forth.

The sentiment certainty visual analysis module 104 can assign sentimentscores to respective attributes in a comment based on use of any ofvarious different sentiment analysis techniques, which involvesidentifying words or phrases in the comment that relate to sentiment(s)expressed by users with respect to each attribute. A sentiment score canbe generated based on the identified words or phrases relating tosentiment(s). The sentiment score provides an indication of whether theexpressed sentiment is positive, negative, or neutral. The sentimentscore can be a numeric score, or alternatively, the sentiment score canhave one of several discrete values (e.g. Positive, Negative, Neutral).

The sentiment result produced by the sentiment certainty visual analysismodule 104 can be associated with different degrees of uncertainty.Certain sentiment results may be more certain than other sentimentresults.

In accordance with some implementations, the sentiment certainty visualanalysis module 104 uses multiple sentiment uncertainty rules 106 toallow for classification of the certainty level (from among multiplecertainty levels 108) associated with each instance of a sentimentanalysis (performed on a respective comment 102). Alternatively, insteadof referring to “certainty levels,” reference can be made to“uncertainty levels.”

In some examples, the sentiment uncertainty rules 106 include threerules, identified as Rule 1, Rule 2, and Rule 3. Rule 1 specifies thatthe presence of an attribute and associated sentiment word (or phrase)follows syntactic natural language patterns. One such example syntacticnatural language pattern is as follows:

This is a nice (adjective) printer (noun).

In the foregoing example, the attribute is “printer,” and the sentimentword associated with this attribute is “nice,” which is an adjective.Thus, the foregoing syntactic natural language pattern indicates that anadjective that appears before a noun typically indicates a sentimentword associated with the attribute.

Another example of syntactic natural language pattern is set forth asfollows:

This printer (noun) is (verb) really (adverb) bad (adjective).

In the foregoing example, the attribute (“printer”) precedes a verb(“is”), which in turn precedes an adverb and/or an adjective. The adverband/or adjective following the verb is indicative of the presence of asentiment word or phrase that is associated with the attribute thatprecedes the verb. Although the foregoing sets forth two examplesyntactic natural language patterns according to Rule 1, note that therecan be other syntactic natural language patterns that are indicative ofthe presence of an attribute and associated sentiment word(s).

Rules 2 and 3 are rules where the language contained in a user commentdoes not follow any of the syntactic natural language patterns. Rule 2specifies that a sentiment word of a given polarity (negative polarityor positive polarity) is close (in proximity) to an attribute. Thesentiment of a given polarity that is “close to” or “in proximity to” anattribute can be defined as follows: the sentiment word(s) and theattribute are located within a given section of the comment (e.g. agiven sentence, a given paragraph, etc.), and/or the sentiment word(s)and attribute are within some predefined number of words of each other.An example of a scenario that satisfies Rule 2 is set forth as follows:

I bought a new printer (noun) and so far I am really satisfied with it.

In the foregoing example, the attribute is “printer,” and a sentiment ofa positive polarity is close (within some predefined number of words) tothe attribute (“really satisfied”), and the attribute and positivepolarity word(s) are located in the same sentence.

Rule 3 specifies the situation where the sentiment word(s) and attributedo not follow any syntactic natural language pattern, and sentiments ofboth polarities (negative polarity and positive polarity) are close orare in proximity to each other (within a given number of words and/orwithin a given section of the comment). An example of such a scenario isset forth as follows:

Some people write bad reviews and other people like everything aboutthis printer (noun).

In the foregoing example, the attribute is “printer,” and there are twosentiments of different polarities (“bad reviews” and “like everything”)expressed in the same sentence.

In a situation that satisfies Rule 3, a distance-weighted mapping can beapplied to derive the overall sentiment associated with the attribute.In the foregoing example, since “like everything” is closer to theattribute “printer” than “bad reviews,” the positive sentiment (“likeeverything”) would be more heavily weighted than the negative sentiment(“bad reviews”).

Although the foregoing sets forth three example sentiment uncertaintyrules, note that in alternative implementations, there can be additionalor different sentiment uncertainty rules.

The different sentiment uncertainty rules are associated with differentlevels of certainty (or different levels of uncertainty). A sentimentresult that satisfies Rule 1 can be assigned a high certainty (or lowuncertainty), a sentiment result that satisfies Rule 2 can be assigned amedium certainty (or medium uncertainty), and a certainty result thatsatisfies Rule 3 can be assigned a low certainty (or high uncertainty).The high certainty, medium certainty, and low certainty can beassociated with respective numeric scores, or alternatively, thedifferent certainty levels can simply be assigned discrete values, suchas “high,” “medium,” and “low.”

There can be special cases where none of the rules are satisfied. Forexample, the attribute can be a sentiment word, such that the sentimentvalue associated with this sentiment word (that makes up the attribute)is ignored. If no sentiment can be identified using any of the foregoingrules, the sentiment result in this scenario is assigned a low certaintylevel.

Another example of special processing involves an attribute that ismentioned multiple times in a sentence. A certainty value of eachinstance can be complemented with an algebraic sign of the correspondingsentiment. A negative sentiment with medium certainty can be assigned anexample value such as −2, while a positive sentiment with high certaintycan be assigned a second value such as +3. The values are thenaggregated (e.g. summed) and the resulting aggregate value can bedecomposed into sentiment and certainty values. In an example, anegative instance of the attribute of medium certainty and a positiveinstance of the attribute of high certainty within a sentence can beevaluated as a positive sentiment result with low certainty (e.g.−2+3=+1).

FIG. 2 depicts a visualization that includes a sequence 202 of comments,where the sequence 202 represents a temporal order of the comments asthey are received. The sequence 202 of comments can also be referred toas a comment sequence track that has a sequence of graphical elements(in the form of general ovals or rectangles with curved ends) eachrepresenting a corresponding comment. The graphical elements in thecomment sequence track 202 are arranged such that they do not overlapeach other.

Although, the graphical elements in the comment sequence track 202 arearranged in sequential temporal order, the exact time points associatedwith the comments are not relevant to the comment sequence track 202.The comment sequence track 202 maintains the temporal order, but removesany space-consuming time gaps and removes overlap of graphical elementsrepresenting respective comments. Thus, in the comment sequence track202, the horizontal axis does not convey exact temporal relations;instead, the same amount of space (equal space) is provided to eachgraphical element in the comment sequence track 202 such that a user canclearly see the arrival sequence of the graphical elements (note thatnone of the graphical elements in the comment sequence track 202 isoccluded by another of the graphical elements).

The graphical elements in the comment sequence track 202 are assigneddifferent colors corresponding to different values of a respectiveattribute of the respective comment. In examples according to FIG. 2,the colors that can be assigned to each of the graphical elements of thecomment sequence track 202 include grey, green, and red. A grey colorassigned to the attribute indicates that the associated sentiment isneutral (not negative and not positive). A green color indicatespositive feedback (e.g. a customer is satisfied with the attribute). Onthe other hand, red represents a negative feedback (e.g. the customer isnot satisfied with the attribute). Graphical elements can also beassigned different (lighter or darker) shades of red or green, toindicate different levels of negative or positive sentiment.

The height of each graphical element in the comment sequence track 202indicates the certainty level of the respective sentiment result that isexpressed by the corresponding color (green, red, or grey). If thesentiment result for a comment has high certainty, then thecorresponding graphical element (such as graphical element 204) has afull height. However, if the sentiment result for the correspondingcomment has a medium certainty, then the corresponding graphical element(such as graphical element 206) has a second height that is less thanthe full height (such as ⅔ of the full height). If the sentiment resultassociated with the comment is low certainty, then the correspondinggraphical element (such as graphical element 208) has a third heightthat is lower than the second height (e.g. ⅓ of the full height).Generally, according to some implementations, a higher certaintyassociated with a sentiment result for a comment is associated with ahigher height of the corresponding graphical element, while a lowercertainty associated with a sentiment result is associated with a lowerheight. In other examples, higher certainty can be indicated with alower height, while lower certainty can be indicated with a higherheight. Even more generally, different sizes of the respective graphicalelements indicate different certainty levels.

In yet further implementations, different visual indicators can be usedto indicate different certainty levels in the comment sequence track202, such as by use of different fill patterns in the graphicalelements.

Since the inter-temporal information (in other words, time gaps betweencomments) has been removed in the comment sequence track 202, a timedensity track 210 is also provided in the visualization 200, in someexamples. The time density track 210 has gap representing elements torepresent time gaps between respective successive comments. In theexample of FIG. 2, the gap representing elements of the time densitytrack 210 includes points along a curve 214. The height of a point alongthe curve 214 represents the time gap between two successive commentsrepresented by two successive graphical elements of the comment sequencetrack 202. The gap representing elements of the time density track 210are aligned with the graphical elements of the comment sequence track202 to allow for easy correlation between the time density track 210 andthe comment sequence track 202.

For example, a point 216 that has a high value indicates that thecomments represented by respective graphical elements 218A and 218B inthe comment sequence track 202 are relatively close to each other intime (and in fact, can overlap each other). Another point 220 that hasan above average height in the time density track 210 indicates that twosuccessive comments represented by graphical elements 218C and 218D inthe comment sequence track 202 are relatively close to each other (theydo not overlap but have a relatively short time gap in between).

Another point 222 having a below average height indicates that the twocorresponding comments represented by two respective graphical elementsin the comment sequence track 202 have a medium time gap between eachother. A zero height of a point along the curve 214 indicates that thereis a relatively long time gap between successive comments.

By looking at the curve 214 of the time density track 210, an analystcan quickly identify points along the comment sequence track 202 thatwould be more interesting (for example, points along the commentsequence track 202 associated with negative feedback and where thecomments are arriving relatively close in time to each other). Such an“interesting” point along the comment sequence track 202 can correspondto times when some problem has occurred, such as a website crashing, aproduct being out of stock, and so forth. Since the graphical elementsof the comment sequence track 202 do not occlude each other, a user cango to any point along the comment sequence track 202 and select, usingan input device, respective ones of the graphical elements to obtainfurther detail (such as in pop-up box 230) regarding the respectivecomments.

Generally, the visualization 200 in FIG. 2 is able to accept interactiveuser input. In this manner, the user can interactively select furtherdetails regarding comments at the individual comment level. Also, bylooking at the combination of the comment sequence track 202 and timedensity track 210, patterns can become more visible to the analyst. Thepattern can be based on colors of the graphical elements of the commentsequence track 202, heights of the graphical elements indicatingrespective certainty levels, and the varying heights of the curve 214 inthe time density track 210.

FIG. 3 illustrates a different type of visualization, in this case acertainty calendar map 300. The certainty calendar map 300 has multiplerows 302, where each row 302 has blocks (or other graphical elements)representing corresponding comments. Each block in a row 302 representsa respective comment. Each row can represent a corresponding timeinterval, such as a corresponding day. In the foregoing example, themultiple rows 302 of the certainty calendar map 300 represent respectivedays of a certain month. In other examples, the different rows 302 canrepresent other time intervals, such as hours, weeks, months, years, andso forth.

The length (corresponding to the number of blocks) of each row 302indicates the number of comments that were received in the respectivetime interval (e.g. day). The longer a row 302, the greater the numberof comments received in that particular day. The color within each blockrepresents a respective sentiment of the comment (green indicatespositive sentiment, grey indicates neutral sentiment, and red indicatesnegative sentiment).

While color assigned to each block indicates the respective sentimentfor the corresponding comment, a different visual indicator, in the formof an amount of filling or some other visual indicator, indicates thecertainty level associated with the respective sentiment result. A fullfilling (in other words the block is completely filled with therespective color) indicates that the associated sentiment result hashigh certainty. A medium filling in the block indicates mediumcertainty, and a low filling in the block indicates low certainty.

In different examples, instead of multiple rows indicating differenttime intervals, the certainty calendar map 300 can be made up ofmultiple columns, where the multiple columns (each having multipleblocks representing respective comments) represent respective timeintervals. More generally, the certainty calendar map 300 has plurallines (rows or columns) each having respective blocks representingcorresponding comments; each of the plural lines represents a respectivetime interval.

FIG. 4 illustrates another type of visualization, in which both acertainty calendar map 400 and a visualization 402 that includes acomment sequence track and time density track (similar to that shown inFIG. 2), are depicted together. The visualization 402 that includes thecomment sequence track and time density track can correspond to one ofthe rows in the certainty calendar map 400.

FIG. 5 illustrates an example in which sentiment result accuracy can beenhanced by using the sentiment uncertainty rules according to someimplementations. A sentiment visualization 502 is produced by asentiment analysis without application of sentiment uncertainty rules asdiscussed above. In the sentiment visualization 502, the differentgraphical elements of a comment sequence track have equal height, sothat an analyst would have no idea what certainty levels are associatedwith the sentiment results of the respective comments.

On the other hand, a sentiment visualization 504 is produced by asentiment analysis with application of uncertainty rules as discussedabove. In the comment sequence track of the sentiment visualization 504,an analyst can easily see certainty levels associated with respectivesentiment results of the respective comments.

FIG. 6 is a flow diagram of a process according to some implementations.The process performs (at 602) sentiment analysis of user feedback (thatincludes multiple comments), where the sentiment analysis uses sentimentuncertainty rules, as discussed above. The process determines (at 604)levels of uncertainty associated with sentiment results of the sentimentanalysis, where the sentiment results identify user sentiments containedin the user feedback.

The process then provides (at 606) at least one visualization of thesentiment results, where the visualization has first visual indicatorsto represent respective user sentiments, and second visual indicatorsassociated with the first visual indicators to represent respectivelevels of uncertainty associated with the respective user sentiments.The at least one visualization provided at 606 can include thevisualization 200 of FIG. 2, the sentiment calendar map 300 of FIG. 3,or the combined visualizations of FIG. 4.

FIG. 7 is a block diagram of an example system that includes a computer700 coupled over a network 702 to various data sources 704. The datasources 704 collect data records (containing user comments) that areentered into the computer 700. The data records can be stored in adatabase 708 in storage media 710.

The computer 700 has a network interface 712 to communicate over thenetwork 702. The network interface 712 is connected to a processor (ormultiple processors) 714. The sentiment certainty visual analysis module104 is executable on the processor(s) 714 to perform the tasks of FIG. 6(and/or other tasks) and to present various visual representations (110)as discussed above in a display device 720.

The sentiment certainty visual analysis module 104 can includemachine-readable instructions that are loaded for execution onprocessor(s) 714. A processor can include a microprocessor,microcontroller, processor module or subsystem, programmable integratedcircuit, programmable gate array, or another control or computingdevice.

The storage media 710 can be implemented as one or multiplecomputer-readable or machine-readable storage media. The storage mediainclude different forms of memory including semiconductor memory devicessuch as dynamic or static random access memories (DRAMs or SRAMs),erasable and programmable read-only memories (EPROMs), electricallyerasable and programmable read-only memories (EEPROMs) and flashmemories; magnetic disks such as fixed, floppy and removable disks;other magnetic media including tape; optical media such as compact disks(CDs) or digital video disks (DVDs); or other types of storage devices.Note that the instructions discussed above can be provided on onecomputer-readable or machine-readable storage medium, or alternatively,can be provided on multiple computer-readable or machine-readablestorage media distributed in a large system having possibly pluralnodes. Such computer-readable or machine-readable storage medium ormedia is (are) considered to be part of an article (or article ofmanufacture). An article or article of manufacture can refer to anymanufactured single component or multiple components.

In the foregoing description, numerous details are set forth to providean understanding of the subject disclosed herein. However,implementations may be practiced without some or all of these details.Other implementations may include modifications and variations from thedetails discussed above. It is intended that the appended claims coversuch modifications and variations.

What is claimed is:
 1. A method of a system having a processor,comprising: performing sentiment analysis of user feedback using aplurality of uncertainty rules, wherein the uncertainty rules include afirst rule indicating that an attribute and an associated sentiment wordor phrase within a comment follow a syntactic natural language pattern,and a second rule indicating that an attribute and an associatedsentiment word or phrase within a comment do not follow the syntacticnatural language pattern; determining levels of uncertainty associatedwith sentiment results of the sentiment analysis, wherein the sentimentresults identify user sentiments contained in the user feedback, andwherein determining the levels of uncertainty comprises: assigning alower level of uncertainty for a first of the sentiment results if thefirst sentiment result satisfies the first rule, and assigning a higherlevel of uncertainty for a second of the sentiment results if the secondsentiment result satisfies the second rule; and providing at least onevisualization of the sentiment results, wherein the visualization hasfirst visual indicators to represent the respective user sentimentsidentified by the sentiment results, and second visual indicatorsassociated with the first visual indicators to represent the respectivelevels of uncertainty associated with the corresponding user sentiments.2. The method of claim 1, wherein providing the at least onevisualization comprises providing a certainty calendar map having linesof graphical elements, where the graphical elements representcorresponding comments in the user feedback, where the lines correspondto respective different time intervals, and a particular one of thelines includes a plurality of graphical elements, at least one of theplurality of graphical elements assigned a first visual indicator torepresent a particular user sentiment in the corresponding comment, andassigned a second visual indicator to represent a level of uncertaintyof the particular user sentiment.
 3. The method of claim 2, wherein thefirst visual indicator of the at least one graphical element is one ofplural different colors that represent different user sentiments.
 4. Themethod of claim 3, wherein the second visual indicator of the at leastone graphical element is one of plural different fillings of the color,wherein the plural different fillings representing respective differentlevels of uncertainty.
 5. The method of claim 1, wherein providing theat least one visualization comprises providing a comment sequence trackhaving graphical elements representing different comments in the userfeedback, wherein a particular one of the graphical elements has a firstvisual indicator representing a user sentiment in the correspondingcomment, and has a second visual indicator representing a level ofuncertainty of the user sentiment.
 6. The method of claim 5, wherein thefirst visual indicator of the particular graphical element is one ofplural different colors that represent different user sentiments.
 7. Themethod of claim 6, wherein the second visual indicator is one of pluraldifferent heights of the particular graphical element, the differentheights representing different levels of uncertainty of the usersentiment.
 8. The method of claim 1, wherein the second rule specifiesthat a sentiment word or phrase of a particular one of multiplepolarities is in a proximity of an attribute.
 9. The method of claim 1,wherein the uncertainty rules further include a third rule specifyingthat sentiment words or phrases of multiple polarities are in proximityto an attribute, and wherein determining the levels of uncertaintyfurther comprises assigning a further higher level of uncertainty for athird of the sentiment results if the third sentiment result satisfiesthe third rule, the further higher level of uncertainty higher than thehigher level of uncertainty assigned for the second sentiment result.10. An article comprising at least one non-transitory computer-readablestorage medium storing instructions that upon execution cause a systemto: apply a plurality of uncertainty rules during performance ofsentiment analysis of user feedback, wherein the uncertainty rulesinclude a first rule indicating that an attribute and an associatedsentiment word or phrase within a comment follow a syntactic naturallanguage pattern, and a second rule indicating that an attribute and anassociated sentiment word or phrase within a comment do not follow thesyntactic natural language pattern; assign levels of uncertainty torespective sentiment results of the sentiment analysis, wherein thesentiment results identify user sentiments contained in the userfeedback, and wherein assigning the levels of uncertainty comprises:assigning a lower level of uncertainty for a first of the sentimentresults if the first sentiment result satisfies the first rule, andassigning a higher level of uncertainty for a second of the sentimentresults if the second sentiment result satisfies the second rule; andpresent at least one visualization of the sentiment results, wherein thevisualization has first visual indicators to represent the respectiveuser sentiments identified by the sentiment results, and second visualindicators associated with the first visual indicators to represent therespective levels of uncertainty associated with the corresponding usersentiments.
 11. The article of claim 10, wherein the visualization hasgraphical elements representing respective comments in the userfeedback, where each of the graphical elements has a corresponding oneof the first visual indicators and a corresponding one of the secondvisual indicators.
 12. The article of claim 11, wherein the first visualindicators include plural different colors representing different usersentiments, including a positive sentiment, a neutral sentiment, and anegative sentiment.
 13. The article of claim 12, wherein the secondvisual indicators include different sizes to represent different levelsof uncertainty.
 14. The article of claim 12, wherein the second visualindicators include different amounts of filling within the graphicalelements to represent different levels of uncertainty.
 15. The articleof claim 10, wherein the visualization includes a certainty calendar maphaving multiple lines, wherein each of the lines include a plurality ofgraphical elements representing respective comments in the userfeedback, and wherein each of the graphical elements is assigned acorresponding one of the first visual indicators and a corresponding oneof the second visual indicators.
 16. The article of claim 10, whereinthe visualization includes a comment sequence track having graphicalelements representing respective comments in the user feedback, whereineach of the graphical elements is assigned a corresponding one of thefirst visual indicators and a corresponding one of the second visualindicators.
 17. A system comprising: a storage medium to store datarecords relating to user feedback; and at least one processor to:perform sentiment analysis of user feedback using a plurality ofuncertainty rules, wherein the uncertainty rules include a first ruleindicating that an attribute and an associated sentiment word or phrasewithin a comment follow a syntactic natural language pattern, and asecond rule indicating that an attribute and an associated sentimentword or phrase within a comment do not follow the syntactic naturallanguage pattern; determine levels of uncertainty associated withsentiment results of the sentiment analysis, wherein the sentimentresults identify user sentiments contained in the user feedback, andwherein determining the levels of uncertainty comprises: assigning alower level of uncertainty for a first of the sentiment results if thefirst sentiment result satisfies the first rule, and assigning a higherlevel of uncertainty for a second of the sentiment results if the secondsentiment result satisfies the second rule; and provide at least onevisualization of the sentiment results, wherein the visualization hasfirst visual indicators to represent the respective user sentimentsidentified by the sentiment results, and second visual indicatorsassociated with the first visual indicators to represent the respectivelevels of uncertainty associated with the respective user sentiments.18. The system of claim 17, wherein the visualization has pluralgraphical elements representing respective ones of comments in the userfeedback, wherein each of the plural graphical elements is assigned acorresponding one of the first visual indicators, and a correspondingone of the second visual indicators.
 19. The system of claim 18, whereinthe first visual indicators include different colors to representdifferent user sentiments, and the second visual indicators are of adifferent type than the first visual indicators.
 20. The article ofclaim 12, wherein the second visual indicators include different heightsof the graphical elements to represent different levels of uncertainty.